The (Un)ideal Physicist: How Humans Rely on Object Interaction for Friction Estimates.

To estimate object properties such as mass or friction, our brain relies on visual information to efficiently compute approximations. The role of sensorimotor feedback, however, is not well understood. Here we tested healthy adults (N = 79) in an inclined-plane problem, that is, how much a plane can be tilted before an object starts to slide, and contrasted the interaction group with observation groups who accessed involved forces by watching objects being manipulated. We created objects of different masses and levels of friction and asked participants to estimate the critical tilt angle after pushing an object, lifting it, or both. Estimates correlated with applied forces and were biased toward object mass, with higher estimates for heavier objects. Our findings highlight that inferences about physical object properties are tightly linked to the human sensorimotor system and that humans integrate sensorimotor information even at the risk of nonveridical perceptual estimates.

Models of vision need some action.

Bowers et al. focus their criticisms on research that compares behavioral and brain data from the ventral stream with a class of deep neural networks for object recognition. While they are right to identify issues with current benchmarking research programs, they overlook a much more fundamental limitation of this literature: Disregarding the importance of action and interaction for perception.

How visuomotor predictability and task demands affect tactile sensitivity on a moving limb during object interaction in a virtual environment.

Tactile sensitivity is decreased on a moving limb compared to the same static limb. This tactile suppression likely reflects an interplay between sensorimotor predictions and sensory feedback. Here, we examined how visuomotor predictability influences tactile suppression. Participants were instructed to hit an approaching virtual object, with the object either never rotating, or always rotating, or rotating unpredictably, prompting related movement adjustments. We probed tactile suppression by delivering a vibrotactile stimulus of varying intensities to the moving hand briefly after the object's rotation and asked participants to indicate if they had felt a vibration. We hypothesized that Unpredictable Rotations would require upweighting of somatosensory feedback from the hand and therefore decrease suppression. Instead, we found stronger suppression with unpredictable than Predictable Rotations. This finding persisted even when visual input from the moving hand was removed and participants had to rely solely on somatosensory feedback of their hand. Importantly, we found a correlation between task demand and tactile suppression in both experiments, indicating that task load can amplify tactile suppression, possibly by downweighting task-irrelevant somatosensory feedback signals to allow for successful task performance when visuomotor task demands are high.

Sensorimotor memories influence movement kinematics but not associated tactile processing.

When interacting with objects, we often rely on visual information. However, vision is not always the most reliable sense for determining relevant object properties. For example, when the mass distribution of an object cannot be inferred visually, humans may rely on predictions about the object's dynamics. Such predictions may not only influence motor behavior but also associated processing of movement-related afferent information, leading to reduced tactile sensitivity during movement. We examined whether predictions based on sensorimotor memories influence grasping kinematics and associated tactile processing. Participants lifted an object of unknown mass distribution and reported whether they detected a tactile stimulus on their grasping hand during the lift. In Experiment 1, the mass distribution could change from trial to trial, whereas in Experiment 2, we intermingled longer with shorter parts of constant and variable mass distributions, while also providing implicit or explicit information about the trial structure. In both experiments, participants grasped the object by predictively choosing contact points that would compensate the mass distribution experienced in the previous trial. Tactile suppression during movement, however, was invariant across conditions. These results suggest that predictions based on sensorimotor memories can influence movement kinematics but not associated tactile perception.

Egocentric cues influence the allocentric spatial memory of object configurations for memory-guided actions.

Allocentric and egocentric reference frames are used to code the spatial position of action targets in reference to objects in the environment, i.e., relative to landmarks (allocentric), or the observer (egocentric). Previous research investigated reference frames in isolation, for example, by shifting landmarks relative to the target and asking participants to reach to the remembered target location. Systematic reaching errors were found in the direction of the landmark shift and used as a proxy for allocentric spatial coding. Here, we examined the interaction of both allocentric and egocentric reference frames by shifting the landmarks as well as the observer. We asked participants to encode a three-dimensional configuration of balls and to reproduce this configuration from memory after a short delay followed by a landmark or an observer shift. We also manipulated the number of landmarks to test its effect on the use of allocentric and egocentric reference frames. We found that participants were less accurate when reproducing the configuration of balls after an observer shift, which was reflected in larger configurational errors. In addition, an increase in the number of landmarks led to a stronger reliance on allocentric cues and a weaker contribution of egocentric cues. In sum, our results highlight the important role of egocentric cues for allocentric spatial coding in the context of memory-guided actions.NEW & NOTEWORTHY Objects in our environment are coded relative to each other (allocentrically) and are thought to serve as independent and reliable cues (landmarks) in the context of unreliable egocentric signals. Contrary to this assumption, we demonstrate that egocentric cues alter the allocentric spatial memory, which could reflect recently discovered interactions between allocentric and egocentric neural processing pathways. Furthermore, additional landmarks lead to a higher contribution of allocentric and a lower contribution of egocentric cues.

Pupil dilation scales with movement distance of real but not of imagined reaching movements.

Pupillary responses have been reliably identified for cognitive and motor tasks, but less is known about their relation to mentally simulated movements (known as motor imagery). Previous work found pupil dilations during the execution of simple finger movements, where peak pupillary dilation scaled with the complexity of the finger movement and force required. Recently, pupillary dilations were reported during imagery of grasping and piano playing. Here, we examined whether pupillary responses are sensitive to the dynamics of the underlying motor task for both executed and imagined reach movements. Participants reached or imagined reaching to one of three targets placed at different distances from a start position. Both executed and imagined movement times scaled with target distance, and they were highly correlated, confirming previous work and suggesting that participants did imagine the respective movement. Increased pupillary dilation was evident during motor execution compared with rest, with stronger dilations for larger movements. Pupil dilations also occurred during motor imagery, however, they were generally weaker than those during motor execution and they were not influenced by imagined movement distance. Instead, dilations during motor imagery resembled pupil responses obtained during a nonmotor imagery task (imagining a previously viewed painting). Our results demonstrate that pupillary responses can reliably capture the dynamics of an executed goal-directed reaching movement, but suggest that pupillary responses during imagined reaching movements reflect general cognitive processes, rather than motor-specific components related to the simulated dynamics of the sensorimotor system.NEW & NOTEWORTHY Pupil size is influenced by the performance of cognitive and motor tasks. Here, we demonstrate that pupil size increases not only during execution but also during mental simulation of goal-directed reaching movements. However, pupil dilations scale with movement amplitude of executed but not of imagined movement, whereas they are similar during motor imagery and a nonmotor imagery task.

Cross-modal metacognition: Visual and tactile confidence share a common scale.

Humans can judge the quality of their perceptual decisions-an ability known as perceptual confidence. Previous work suggested that confidence can be evaluated on an abstract scale that can be sensory modality-independent or even domain-general. However, evidence is still scarce on whether confidence judgments can be directly made across visual and tactile decisions. Here, we investigated in a sample of 56 adults whether visual and tactile confidence share a common scale by measuring visual contrast and vibrotactile discrimination thresholds in a confidence-forced choice paradigm. Confidence judgments were made about the correctness of the perceptual decision between two trials involving either the same or different modalities. To estimate confidence efficiency, we compared discrimination thresholds obtained from all trials to those from trials judged to be relatively more confident. We found evidence for metaperception because higher confidence was associated with better perceptual performance in both modalities. Importantly, participants were able to judge their confidence across modalities without any costs in metaperceptual sensitivity and only minor changes in response times compared to unimodal confidence judgments. In addition, we were able to predict cross-modal confidence well from unimodal judgments. In conclusion, our findings show that perceptual confidence is computed on an abstract scale and that it can assess the quality of our decisions across sensory modalities.

vexptoolbox: A software toolbox for human behavior studies using the Vizard virtual reality platform.

Virtual reality (VR) is a powerful tool for researchers due to its potential to study dynamic human behavior in highly naturalistic environments while retaining full control over the presented stimuli. Due to advancements in consumer hardware, VR devices are now very affordable and have also started to include technologies such as eye tracking, further extending potential research applications. Rendering engines such as Unity, Unreal, or Vizard now enable researchers to easily create complex VR environments. However, implementing the experimental design can still pose a challenge, and these packages do not provide out-of-the-box support for trial-based behavioral experiments. Here, we present a Python toolbox, designed to facilitate common tasks when developing experiments using the Vizard VR platform. It includes functionality for common tasks like creating, randomizing, and presenting trial-based experimental designs or saving results to standardized file formats. Moreover, the toolbox greatly simplifies continuous recording of eye and body movements using any hardware supported in Vizard. We further implement and describe a simple goal-directed reaching task in VR and show sample data recorded from five volunteers. The toolbox, example code, and data are all available on GitHub under an open-source license. We hope that our toolbox can simplify VR experiment development, reduce code duplication, and aid reproducibility and open-science efforts.

Episodic Binding and Retrieval in Sequences of Discrete Movements - Evidence from Grasping Actions.

In everyday life humans are confronted with changing environmental demands. In order to act successfully and achieve intended goals, action control is required. A recent approach, the Binding and Retrieval in Action Control (BRAC) framework, attempts to provide an overarching perspective on action control. Based on basic principles such as binding and retrieval, findings from several experimental paradigms could be integrated. However, the focus so far has been on rather artificial paradigms involving very simple motor response requirements, like finger lifting or button presses. We aimed to extend the BRAC framework to more complex movements consisting of a sequence of several discrete actions. Participants were asked to grasp and lift an object with an uneven mass distribution. Object features, like mass distribution and position, were either kept constant on a global level or varied in a pseudorandomized manner. When both object features were kept constant, participants were able to adjust their grasp so that it resulted in a more stable lift and less object roll. Further, with randomly mixed object features, we found best task performance when both object features were completely repeated from one trial to the other. These results suggest that tasks with more complex movements are capable of reflecting principles of action control as defined by the BRAC framework. This offers the possibility to test these principles in even more complex and ecologically relevant paradigms to improve our understanding of everyday life actions.

Demand characteristics challenge effects in embodiment and presence.

The sensations to own and control a body as well as being located in a body describe the relation between ourselves and our body, termed embodiment. Embodiment plays a central role in our everyday actions. However, its assessment is challenging. Recent findings suggest that measures on embodiment are confounded by demand characteristics and suggestibility. To investigate the impact of demand characteristics on embodiment and presence, we compared results from an online experiment measuring participants' expectations, to the same experiment in virtual reality (VR). In the online experiment, participants watched a video of a participant performing the VR experiment. In the VR experiment, participants performed a soap-bubble-kicking task, which allowed the feelings of embodiment and presence to arise. We manipulated temporo-spatial movement synchrony (Movement: synchronous, asynchronous) and avatar visibility (Visibility: visible, invisible). In addition, we assessed participants' suggestibility with exercises. The introduced manipulations influenced the ratings in both experiments similarly. Embodiment ratings were additionally affected by suggestibility. Altogether, our results show that participants were aware of the research hypotheses, which indicates that demand characteristics confound embodiment and presence research alike. Overcoming challenges of demand characteristics is crucial to correctly interpret scientific results and to translate these results into applied settings.

Tactile suppression stems from specific sensorimotor predictions.

The ability to sample sensory information with our hands is crucial for smooth and efficient interactions with the world. Despite this important role of touch, tactile sensations on a moving hand are perceived weaker than when presented on the same but stationary hand. This phenomenon of tactile suppression has been explained by predictive mechanisms, such as internal forward models, that estimate future sensory states of the body on the basis of the motor command and suppress the associated predicted sensory feedback. The origins of tactile suppression have sparked a lot of debate, with contemporary accounts claiming that suppression is independent of sensorimotor predictions and is instead due to an unspecific mechanism. Here, we target this debate and provide evidence for specific tactile suppression due to precise sensorimotor predictions. Participants stroked with their finger over textured objects that caused predictable vibrotactile feedback signals on that finger. Shortly before touching the texture, we probed tactile suppression by applying external vibrotactile probes on the moving finger that either matched or mismatched the frequency generated by the stroking movement along the texture. We found stronger suppression of the probes that matched the predicted sensory feedback. These results show that tactile suppression is specifically tuned to the predicted sensory states of a movement.

The role of grasping demands on tactile suppression.

Tactile perception is impaired in a limb that is moving compared to when it is static. A possible mechanism that explains this phenomenon is an internal forward model that estimates future sensory states of the moving limb and suppresses associated feedback signals arising from that limb. Because sensorimotor estimations are based on an interplay of efferent and afferent feedback signals, the strength of tactile suppression may also depend on the relative utilization of sensory feedback from the moving limb. To test how the need to process somatosensory feedback influences movement-induced tactile suppression, we asked participants to perform reach-to-grasp movement of different demands: the target object was covered with materials of different frictional properties and the task was performed both with and without vision. As expected, participants performed the grasping movement more carefully when interacting with objects of low than high friction surfaces and when performing the task without vision. This denotes a greater need for somatosensory guidance of the digits to appropriately position the digits on the object. Accordingly, tactile suppression was weaker when grasping low than high friction objects, but only when grasping without vision. This suggests that movement-induced tactile suppression is modulated by grasping demands. Tactile suppression is downregulated when the need to process somatosensory feedback signals from that moving limb increases, like in situations when somatosensory input about the digit's state is the sole source of sensory information and when this information is particularly important for the task at hand.

Linking Signal Relevancy and Intensity in Predictive Tactile Suppression.

Predictable somatosensory feedback leads to a reduction in tactile sensitivity. This phenomenon, called tactile suppression, relies on a mechanism that uses an efference copy of motor commands to help select relevant aspects of incoming sensory signals. We investigated whether tactile suppression is modulated by (a) the task-relevancy of the predicted consequences of movement and (b) the intensity of related somatosensory feedback signals. Participants reached to a target region in the air in front of a screen; visual or tactile feedback indicated the reach was successful. Furthermore, tactile feedback intensity (strong vs. weak) varied across two groups of participants. We measured tactile suppression by comparing detection thresholds for a probing vibration applied to the finger either early or late during reach and at rest. As expected, we found an overall decrease in late-reach suppression, as no touch was involved at the end of the reach. We observed an increase in the degree of tactile suppression when strong tactile feedback was given at the end of the reach, compared to when weak tactile feedback or visual feedback was given. Our results suggest that the extent of tactile suppression can be adapted to different demands of somatosensory processing. Downregulation of this mechanism is invoked only when the consequences of missing a weak movement sequence are severe for the task. The decisive factor for the presence of tactile suppression seems not to be the predicted action effect as such, but the need to detect and process anticipated feedback signals occurring during movement.

Eye Tracking in Virtual Reality: Vive Pro Eye Spatial Accuracy, Precision, and Calibration Reliability.

A growing number of virtual reality devices now include eye tracking technology, which can facilitate oculomotor and cognitive research in VR and enable use cases like foveated rendering. These applications require different tracking performance, often measured as spatial accuracy and precision. While manufacturers report data quality estimates for their devices, these typically represent ideal performance and may not reflect real-world data quality. Additionally, it is unclear how accuracy and precision change across sessions within the same participant or between devices, and how performance is influenced by vision correction. Here, we measured spatial accuracy and precision of the Vive Pro Eye built-in eye tracker across a range of 30 visual degrees horizontally and vertically. Participants completed ten measurement sessions over multiple days, allowing to evaluate calibration reliability. Accuracy and precision were highest for central gaze and decreased with greater eccentricity in both axes. Calibration was successful in all participants, including those wearing contacts or glasses, but glasses yielded significantly lower performance. We further found differences in accuracy (but not precision) between two Vive Pro Eye headsets, and estimated participants' inter-pupillary distance. Our metrics suggest high calibration reliability and can serve as a baseline for expected eye tracking performance in VR experiments.

Different contributions of efferent and reafferent feedback to sensorimotor temporal recalibration.

Adaptation to delays between actions and sensory feedback is important for efficiently interacting with our environment. Adaptation may rely on predictions of action-feedback pairing (motor-sensory component), or predictions of tactile-proprioceptive sensation from the action and sensory feedback of the action (inter-sensory component). Reliability of temporal information might differ across sensory feedback modalities (e.g. auditory or visual), which in turn influences adaptation. Here, we investigated the role of motor-sensory and inter-sensory components on sensorimotor temporal recalibration for motor-auditory (button press-tone) and motor-visual (button press-Gabor patch) events. In the adaptation phase of the experiment, action-feedback pairs were presented with systematic temporal delays (0 ms or 150 ms). In the subsequent test phase, audio/visual feedback of the action were presented with variable delays. The participants were then asked whether they detected a delay. To disentangle motor-sensory from inter-sensory component, we varied movements (active button press or passive depression of button) at adaptation and test. Our results suggest that motor-auditory recalibration is mainly driven by the motor-sensory component, whereas motor-visual recalibration is mainly driven by the inter-sensory component. Recalibration transferred from vision to audition, but not from audition to vision. These results indicate that motor-sensory and inter-sensory components contribute to recalibration in a modality-dependent manner.

Further Evidence That People Rely on Egocentric Information to Guide a Cursor to a Visible Target.

Everyday movements are guided by objects' positions relative to other items in the scene (allocentric information) as well as by objects' positions relative to oneself (egocentric information). Allocentric information can guide movements to the remembered positions of hidden objects, but is it also used when the object remains visible? To stimulate the use of allocentric information, the position of the participant's finger controlled the velocity of a cursor that they used to intercept moving targets, so there was no one-to-one mapping between egocentric positions of the hand and cursor. We evaluated whether participants relied on allocentric information by shifting all task-relevant items simultaneously leaving their allocentric relationships unchanged. If participants rely on allocentric information they should not respond to this perturbation. However, they did. They responded in accordance with their responses to each item shifting independently, supporting the idea that fast guidance of ongoing movements primarily relies on egocentric information.

Reach-relevant somatosensory signals modulate activity in the tactile suppression network.

Somatosensory signals on a moving limb are typically suppressed. This results mainly from a predictive mechanism that generates an efference copy, and attenuates the predicted sensory consequences of that movement. Sensory feedback is, however, important for movement control. Behavioral studies show that the strength of suppression on a moving limb increases during somatosensory reaching, when reach-relevant somatosensory signals from the target limb can be additionally used to plan and guide the movement, leading to increased reliability of sensorimotor predictions. It is still unknown how this suppression is neurally implemented. In this fMRI study, participants reached to a somatosensory (static finger) or an external target (touch-screen) without vision. To probe suppression, participants detected brief vibrotactile stimuli on their moving finger shortly before reach onset. As expected, sensitivity to probes was reduced during reaching compared to baseline (resting), and this suppression was stronger during somatosensory than external reaching. BOLD activation associated with suppression was also modulated by the reach target: relative to baseline, processing of probes during somatosensory reaching led to distinct BOLD deactivations in somatosensory regions (postcentral gyrus, supramarginal gyrus-SMG) whereas probes during external reaching led to deactivations in the cerebellum. In line with the behavioral results, we also found additional deactivations during somatosensory relative to external reaching in the supplementary motor area, a region linked with sensorimotor prediction. Somatosensory reaching was also linked with increased functional connectivity between the left SMG and the right parietal operculum along with the right anterior insula. We show that somatosensory processing on a moving limb is reduced when additional reach-relevant feedback signals from the target limb contribute to the movement, by down-regulating activation in regions associated with predictive and feedback processing.

Dynamic temporal modulation of somatosensory processing during reaching.

Sensorimotor control of human action integrates feedforward policies that predict future body states with online sensory feedback. These predictions lead to a suppression of the associated feedback signals. Here, we examine whether somatosensory processing throughout a goal-directed movement is constantly suppressed or dynamically tuned so that online feedback processing is enhanced at critical moments of the movement. Participants reached towards their other hand in the absence of visual input and detected a probing tactile stimulus on their moving or static hand. Somatosensory processing on the moving hand was dynamically tuned over the time course of reaching, being hampered in early and late stages of the movement, but, interestingly, recovering around the time of maximal speed. This novel finding of temporal somatosensory tuning was further corroborated in a second experiment, in which larger movement amplitudes shifted the absolute time of maximal speed later in the movement. We further show that the release from suppression on the moving limb was temporally coupled with enhanced somatosensory processing on the target hand. We discuss these results in the context of optimal feedforward control and suggest that somatosensory processing is dynamically tuned during the time course of reaching by enhancing sensory processing at critical moments of the movement.

The influence of afferent input on somatosensory suppression during grasping.

The processing of somatosensory information is hampered on a moving limb. This suppression has been widely attributed to sensorimotor predictions that suppress the associated feedback, though postdictive mechanisms are also involved. Here, we investigated the extent to which suppression on a limb is influenced by backward somatosensory signals, such as afferents associated with forces that this limb applies. Participants grasped and lifted objects of symmetric and asymmetric mass distributions using a precision grip. We probed somatosensory processing at the moment of the grasp by presenting a vibrotactile stimulus either on the thumb or index finger and asked participants to report if they felt this stimulus. Participants applied greater forces with the thumb and index finger for objects loaded to the thumb's or index finger's endpoint, respectively. However, suppression was not influenced by the different applied forces. Suppression on the digits remained constant both when grasping heavier objects, and thus applying even greater forces, and when probing suppression on the skin over the muscle that controlled force application. These results support the idea that somatosensory suppression is predictive in nature while backward masking may only play a minor role in somatosensory processing on the moving hand, at least in this context.